Bot Interaction Detection

Note: This feature is available on request.

Introduction

Email marketers using Marigold products rely on hidden pixels within emails to make callbacks to determine when an email is opened. This provides useful tracking information to better understand the effectiveness of marketing strategies. However, open rates and other metrics can be disrupted by services(*) such as Apple Mail Privacy Detection (MPP) and other non-human clicks, which are designed to automatically open and screen emails on the user's behalf.

(*) Note: Many recipients are reading emails on devices or corporate networks protected by anti-malware software, enterprise security and monitoring software, browser extensions, or antivirus firewalls. When these subscribers receive an email or interact by clicking on a link, these protective systems will investigate the email and at times click on some - or all, in extreme cases - of the links in the email.

These services often pre-fetch content and hide the recipient's IP address and related information, which disrupts metrics used for tracking open rates and impacts downstream activities like list segmentation.
Not all customers nor all messages are impacted, but more extreme cases can result in inflating the number of total clicks by up to 10 times.
Protecting people from malware and bad actors is an important job, but email senders can find that their email reporting data is significantly affected by these systems, compromising their ability to interpret results and make business decisions.

Therefore, Bot Interaction Detection is integrated within Engage/Campaign to aid in identifying bot activity.

It currently has an impact to filter data for Generic Exports.
In the future, more functionality will be affected by Bot Interaction Detection such as Journey Reports, Dashboards, Apple Mail Privacy Detection (MPP), List Segmentation/Audience Filters, EMIT and Send-Time Optimization.

 

How are bots detected?

Marigold uses a growing combination of industry-standard and proprietary techniques to detect bots.
Currently, for each incoming request, the following are validated :

  • IP address — A certain range of IP addresses are allocated to bots. We do a detection of the IP address that's being used.

  • User Agent — Specific bots use specific user agents (for example Googlebot).

If the combination is an incoming IP address being within a bot IP range, and a user agent that's part of the list we consider to be assigned to a bot, it's considered to be a bot within Engage.

 

Impacted Engage functionality

Bot Interaction detection has an impact on Generic Exports.
The exported data contains a field indicating if it's a real (human) interaction or a bot interaction.

Note: Generic Exports contain a uniform set of system usage data that can be delivered by Marigold to their Engage platform users.
These files will make it possible to get the history of communications and interactions done with their install.
More info on Generic Exports can be found after logging into the MyEngage Portal.

Technical Note: For bot detection in Generic Exports, two optional fields are used in the Interaction table.
- IS_POTENTIAL_BOT : Flagging potential bot activity.
- BOT_DETECTION_TYPE : Describes how the bot was detected.
Note that these fields are not available by default, and can be added on request through a Support ticket.